Markov Processes
نویسنده
چکیده
1 General Properties of Markov Processes 2 1.1 Discrete Time Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.1.1 Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.1.2 Steady State . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 1.2 Continuous Time Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 1.2.1 Waiting Times and Discrete Time Embedding . . . . . . . . . . . . . . . . . . . . . . . 8 1.2.2 Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 1.3 Spectral Expansions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 1.4 Detailed Balance and Reversibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 1.4.1 Self-Adjoint Symmetry and Spectral Decomposition . . . . . . . . . . . . . . . . . . . 11 1.4.2 Steady State Computation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 1.4.3 Detailed Balance in Physical Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 1.5 Entropy and the Approach to Steady State . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 1.6 The Backward Equation and Observables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 1.7 Martingales . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 1.8 The Deterministic Limit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
منابع مشابه
On $L_1$-weak ergodicity of nonhomogeneous continuous-time Markov processes
In the present paper we investigate the $L_1$-weak ergodicity of nonhomogeneous continuous-time Markov processes with general state spaces. We provide a necessary and sufficient condition for such processes to satisfy the $L_1$-weak ergodicity. Moreover, we apply the obtained results to establish $L_1$-weak ergodicity of quadratic stochastic processes.
متن کاملON THE GENERALIZATION OF N-PLE MARKOV PROCESSES
The notion of N-ple Markov process is defined in a quite general framework and it is shown that N-ple Markov processes-arel inear combinationso f some martingales
متن کاملApplication of Markov Processes to the Machine Delays Analysis
Production and non-productive equipment and personnel delays are a critical element of any production system. The frequency and length of delays impact heavily on the production and economic efficiency of these systems. Machining processes in wood industry are particularly vulnerable to productive and non-productive delays. Whereas, traditional manufacturing industries usually operate on homoge...
متن کاملADK Entropy and ADK Entropy Rate in Irreducible- Aperiodic Markov Chain and Gaussian Processes
In this paper, the two parameter ADK entropy, as a generalized of Re'nyi entropy, is considered and some properties of it, are investigated. We will see that the ADK entropy for continuous random variables is invariant under a location and is not invariant under a scale transformation of the random variable. Furthermore, the joint ADK entropy, conditional ADK entropy, and chain rule of this ent...
متن کاملExtended Geometric Processes: Semiparametric Estimation and Application to ReliabilityImperfect repair, Markov renewal equation, replacement policy
Lam (2007) introduces a generalization of renewal processes named Geometric processes, where inter-arrival times are independent and identically distributed up to a multiplicative scale parameter, in a geometric fashion. We here envision a more general scaling, not necessar- ily geometric. The corresponding counting process is named Extended Geometric Process (EGP). Semiparametric estimates are...
متن کامل